Operator norm

 ( Functional Analysis ) 

In mathematics, the operator norm measures the "size" of certain by assigning each a real number called its . Formally, it is a norm defined on the space of bounded linear operators between two given normed vector spaces. Informally, the operator norm $\backslash |T\backslash |$ of a linear map $T\; :\; X\; \backslash to\; Y$ is the maximum factor by which it "lengthens" vectors.

---

Introduction and definition Given two normed vector spaces $V$ and $W$ (over the same base field, either the $\backslash R$ or the $\backslash Complex$), a linear map $A\; :\; V\; \backslash to\; W$ is continuous if and only if there exists a real number $c$ such that $$\backslash |Av\backslash |\; \backslash leq\; c\; \backslash |v\backslash |\; \backslash quad\; \backslash text\{\; for\; all\; \}\; v\backslash in\; V.$$

The norm on the left is the one in $W$ and the norm on the right is the one in $V$. Intuitively, the continuous operator $A$ never increases the length of any vector by more than a factor of $c.$ Thus the image of a bounded set under a continuous operator is also bounded. Because of this property, the continuous linear operators are also known as . In order to "measure the size" of $A,$ one can take the infimum of the numbers $c$ such that the above inequality holds for all $v\; \backslash in\; V.$ This number represents the maximum scalar factor by which $A$ "lengthens" vectors. In other words, the "size" of $A$ is measured by how much it "lengthens" vectors in the "biggest" case. So we define the operator norm of $A$ as $$\backslash |A\backslash |\_\backslash text\{op\}\; =\; \backslash inf\backslash \{\; c\; \backslash geq\; 0\; :\; \backslash |Av\backslash |\; \backslash leq\; c\; \backslash |v\backslash |\; \backslash text\{\; for\; all\; \}\; v\; \backslash in\; V\; \backslash \}.$$

The infimum is attained as the set of all such $c$ is Closed set, Empty set, and Bounded set from below.See e.g. Lemma 6.2 of .

It is important to bear in mind that this operator norm depends on the choice of norms for the normed vector spaces $V$ and $W$.

---

Examples Every real $m$-by-$n$ matrix corresponds to a linear map from $\backslash R^n$ to $\backslash R^m.$ Each pair of the plethora of (vector) norms applicable to real vector spaces induces an operator norm for all $m$-by-$n$ matrices of real numbers; these induced norms form a subset of .

If we specifically choose the Euclidean norm on both $\backslash R^n$ and $\backslash R^m,$ then the matrix norm given to a matrix $A$ is the square root of the largest eigenvalue of the matrix $A^\{*\}\; A$ (where $A^\{*\}$ denotes the conjugate transpose of $A$). This is equivalent to assigning the largest singular value of $A.$

Passing to a typical infinite-dimensional example, consider the sequence space $\backslash ell^2,$ which is an Lp space, defined by

This can be viewed as an infinite-dimensional analogue of the Euclidean space $\backslash Complex^n.$ Now consider a bounded sequence $s\_\{\backslash bull\}\; =\; \backslash left(s\_n\backslash right)\_\{n=1\}^\backslash infty.$ The sequence $s\_\{\backslash bull\}$ is an element of the space $\backslash ell^\backslash infty,$ with a norm given by $$\backslash left\backslash |s\_\{\backslash bull\}\backslash right\backslash |\_\backslash infty\; =\; \backslash sup\; \_n\; \backslash left|s\_n\backslash right|.$$

Define an operator $T\_s$ by pointwise multiplication: $$\backslash left(a\_n\backslash right)\_\{n=1\}^\{\backslash infty\}\; \backslash ;\backslash stackrel\{T\_s\}\{\backslash mapsto\}\backslash ;\backslash \; \backslash left(s\_n\; \backslash cdot\; a\_n\backslash right)\_\{n=1\}^\{\backslash infty\}.$$

The operator $T\_s$ is bounded with operator norm $$\backslash left\backslash |T\_s\backslash right\backslash |\_\backslash text\{op\}\; =\; \backslash left\backslash |s\_\{\backslash bull\}\backslash right\backslash |\_\backslash infty.$$

This discussion extends directly to the case where $\backslash ell^2$ is replaced by a general $L^p$ space with $p\; >\; 1$ and $\backslash ell^\backslash infty$ replaced by $L^\backslash infty.$

---

Equivalent definitions Let $A\; :\; V\; \backslash to\; W$ be a linear operator between normed spaces. The first four definitions are always equivalent, and if in addition $V\; \backslash neq\; \backslash \{0\backslash \}$ then they are all equivalent:

$$

\begin{alignat}{4} \|A\|_\text{op} &= \inf &&\{ c \geq 0 ~&&:~ \| A v \| \leq c \| v \| ~&&~ \text{ for all } ~&&v \in V \} \\ &= \sup &&\{ \| Av \| ~&&:~ \| v \| \leq 1 ~&&~\mbox{ and } ~&&v \in V \} \\ &= \sup &&\{ \| Av \| ~&&:~ \| v \| < 1 ~&&~\mbox{ and } ~&&v \in V \} \\ &= \sup &&\{ \| Av \| ~&&:~ \| v \| \in \{0,1\} ~&&~\mbox{ and } ~&&v \in V \} \\ &= \sup &&\{ \| Av \| ~&&:~ \| v \| = 1 ~&&~\mbox{ and } ~&&v \in V \} \;\;\;\text{ this equality holds if and only if } V \neq \{ 0 \} \\ &= \sup &&\bigg\{ \frac{\| Av \|}{\| v \|} ~&&:~ v \ne 0 ~&&~\mbox{ and } ~&&v \in V \bigg\} \;\;\;\text{ this equality holds if and only if } V \neq \{ 0 \}. \\ \end{alignat} If $V\; =\; \backslash \{0\backslash \}$ then the sets in the last two rows will be empty, and consequently their over the set $-\backslash infty,$ will equal $-\backslash infty$ instead of the correct value of $0.$ If the supremum is taken over the set $0,$ instead, then the supremum of the empty set is $0$ and the formulas hold for any $V.$

Importantly, a linear operator $A\; :\; V\; \backslash to\; W$ is not, in general, guaranteed to achieve its norm $\backslash |A\backslash |\_\backslash text\{op\}\; =\; \backslash sup\; \backslash \{\backslash |A\; v\backslash |\; :\; \backslash |v\backslash |\; \backslash leq\; 1,\; v\; \backslash in\; V\backslash \}$ on the closed unit ball $\backslash \{v\; \backslash in\; V\; :\; \backslash |v\backslash |\; \backslash leq\; 1\backslash \},$ meaning that there might not exist any vector $u\; \backslash in\; V$ of norm $\backslash |u\backslash |\; \backslash leq\; 1$ such that $\backslash |A\backslash |\_\backslash text\{op\}\; =\; \backslash |A\; u\backslash |$ (if such a vector does exist and if $A\; \backslash neq\; 0,$ then $u$ would necessarily have unit norm $\backslash |u\backslash |\; =\; 1$). R.C. James proved James's theorem in 1964, which states that a Banach space $V$ is reflexive space if and only if every bounded linear functional $f\; \backslash in\; V^*$ achieves its Dual norm on the closed unit ball. It follows, in particular, that every non-reflexive Banach space has some bounded linear functional (a type of bounded linear operator) that does not achieve its norm on the closed unit ball.

If $A\; :\; V\; \backslash to\; W$ is bounded then $$\backslash |A\backslash |\_\backslash text\{op\}\; =\; \backslash sup\; \backslash left\backslash \{\backslash left|w^*(A\; v)\backslash right|\; :\; \backslash |v\backslash |\; \backslash leq\; 1,\; \backslash left\backslash |w^*\backslash right\backslash |\; \backslash leq\; 1\; \backslash text\{\; where\; \}\; v\; \backslash in\; V,\; w^*\; \backslash in\; W^*\backslash right\backslash \}$$ and $$\backslash |A\backslash |\_\backslash text\{op\}\; =\; \backslash left\backslash |\{\}^tA\backslash right\backslash |\_\backslash text\{op\}$$ where $\{\}^t\; A\; :\; W^*\; \backslash to\; V^*$ is the transpose of $A\; :\; V\; \backslash to\; W,$ which is the linear operator defined by $w^*\; \backslash ,\backslash mapsto\backslash ,\; w^*\; \backslash circ\; A.$

---

Properties The operator norm is indeed a norm on the space of all between $V$ and $W$. This means $$\backslash |A\backslash |\_\backslash text\{op\}\; \backslash geq\; 0\; \backslash mbox\{\; and\; \}\; \backslash |A\backslash |\_\backslash text\{op\}\; =\; 0\; \backslash mbox\{\; if\; and\; only\; if\; \}\; A\; =\; 0,$$ $$\backslash |aA\backslash |\_\backslash text\{op\}\; =\; |a|\; \backslash |A\backslash |\_\backslash text\{op\}\; \backslash mbox\{\; for\; every\; scalar\; \}\; a\; ,$$ $$\backslash |A\; +\; B\backslash |\_\backslash text\{op\}\; \backslash leq\; \backslash |A\backslash |\_\backslash text\{op\}\; +\; \backslash |B\backslash |\_\backslash text\{op\}.$$

The following inequality is an immediate consequence of the definition: $$\backslash |Av\backslash |\; \backslash leq\; \backslash |A\backslash |\_\backslash text\{op\}\; \backslash |v\backslash |\; \backslash \; \backslash mbox\{\; for\; every\; \}\backslash \; v\; \backslash in\; V.$$

The operator norm is also compatible with the composition, or multiplication, of operators: if $V$, $W$ and $X$ are three normed spaces over the same base field, and $A\; :\; V\; \backslash to\; W$ and $B\; :\; W\; \backslash to\; X$ are two bounded operators, then it is a sub-multiplicative norm, that is: $$\backslash |BA\backslash |\_\backslash text\{op\}\; \backslash leq\; \backslash |B\backslash |\_\backslash text\{op\}\; \backslash |A\backslash |\_\backslash text\{op\}.$$

For bounded operators on $V$, this implies that operator multiplication is jointly continuous.

It follows from the definition that if a sequence of operators converges in operator norm, it converges uniformly on bounded sets.

---

Table of common operator norms By choosing different norms for the codomain, used in computing $\backslash |Av\backslash |$, and the domain, used in computing $\backslash |v\backslash |$, we obtain different values for the operator norm. Some common operator norms are easy to calculate, and others are NP-hard. Except for the NP-hard norms, all these norms can be calculated in $N^2$ operations (for an $N\; \backslash times\; N$ matrix), with the exception of the $\backslash ell\_2\; -\; \backslash ell\_2$ norm (which requires $N^3$ operations for the exact answer, or fewer if you approximate it with the Power iteration or Lanczos iterations).

+ Computability of Operator Normssection 4.3.1, Joel Tropp's PhD thesis, [1]

The norm of the adjoint or transpose can be computed as follows. We have that for any $p,\; q,$ then $\backslash |A\backslash |\_\{p\backslash rightarrow\; q\}\; =\; \backslash |A^*\backslash |\_\{q\text{'}\backslash rightarrow\; p\text{'}\}$ where $p\text{'},\; q\text{'}$ are Hölder conjugate to $p,\; q,$ that is, $1/p\; +\; 1/p\text{'}\; =\; 1$ and $1/q\; +\; 1/q\text{'}\; =\; 1.$

---

Operators on a Hilbert space Suppose $H$ is a real or complex Hilbert space. If $A\; :\; H\; \backslash to\; H$ is a bounded linear operator, then we have $$\backslash |A\backslash |\_\backslash text\{op\}\; =\; \backslash left\backslash |A^*\backslash right\backslash |\_\backslash text\{op\}$$ and $$\backslash left\backslash |A^*\; A\backslash right\backslash |\_\backslash text\{op\}\; =\; \backslash |A\backslash |\_\backslash text\{op\}^2,$$ where $A^\{*\}$ denotes the adjoint operator of $A$ (which in with the standard inner product corresponds to the conjugate transpose of the matrix $A$).

In general, the spectral radius of $A$ is bounded above by the operator norm of $A$: $$\backslash rho(A)\; \backslash leq\; \backslash |A\backslash |\_\backslash text\{op\}.$$

To see why equality may not always hold, consider the Jordan canonical form of a matrix in the finite-dimensional case. Because there are non-zero entries on the superdiagonal, equality may be violated. The quasinilpotent operators is one class of such examples. A nonzero quasinilpotent operator $A$ has spectrum $\backslash \{0\backslash \}.$ So $\backslash rho(A)\; =\; 0$ while $\backslash |A\backslash |\_\backslash text\{op\}\; >\; 0.$

However, when a matrix $N$ is Normal matrix, its Jordan canonical form is diagonal (up to unitary equivalence); this is the spectral theorem. In that case it is easy to see that $$\backslash rho(N)\; =\; \backslash |N\backslash |\_\backslash text\{op\}.$$

This formula can sometimes be used to compute the operator norm of a given bounded operator $A$: define the Hermitian operator $B\; =\; A^\{*\}\; A,$ determine its spectral radius, and take the square root to obtain the operator norm of $A.$

The space of bounded operators on $H,$ with the topology induced by operator norm, is not Separable space. For example, consider the Lp space $L^20,,$ which is a Hilbert space. For let $\backslash Omega\_t$ be the characteristic function of $0,,$ and $P\_t$ be the multiplication operator given by $\backslash Omega\_t,$ that is, $$P\_t\; (f)\; =\; f\; \backslash cdot\; \backslash Omega\_t.$$

Then each $P\_t$ is a bounded operator with operator norm 1 and $$\backslash left\backslash |P\_t\; -\; P\_s\backslash right\backslash |\_\backslash text\{op\}\; =\; 1\; \backslash quad\; \backslash mbox\{\; for\; all\; \}\; \backslash quad\; t\; \backslash neq\; s.$$

But is an uncountable set. This implies the space of bounded operators on $L^2(0,)$ is not separable, in operator norm. One can compare this with the fact that the sequence space $\backslash ell^\{\backslash infty\}$ is not separable.

The associative algebra of all bounded operators on a Hilbert space, together with the operator norm and the adjoint operation, yields a C*-algebra.

---

See also

---

Notes

• .
• Joe Diestel (1984). 9780387908595, Springer-Verlag. ISBN 9780387908595

Post Comment